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Abstract 

The  purpose  of  this  study  is  to  extend  the  basic  research  findings  on  spatial  representations 
to  a  unique  application  area  and  extend  the  small  number  of  applied  research  studies  in  this 
general  area  to  a  more  robust,  Navy-specific  application  set  (i.e..  Unmanned  Aerial  Systems).  In 
this  study,  participants  assumed  the  role  of  a  sensor  operator  for  an  unmanned  aerial  system 
(UAS)  simulation  while  riding  in  the  fuselage  of  an  airborne  Lockheed  P-3  Orion.  The  P-3  flew 
a  flight  profile  of  intermittent  ascending,  descending,  and  turning  profiles  (in  strict  accordance 
with  an  emphasis  on  safety  of  flight)  to  induce  a  maximum  level  of  spatial  discordance  to  the 
sensor  operator  screen  where  the  participant  tracked  simulated  targets.  Participants  also 
performed  trials  on  the  ground  with  the  laptop-based  UAS  sensor  operator  simulation  to  establish 
baseline  performance.  In  a  counterbalanced  design,  the  participants  performed  trials  while 
airborne  and  on  the  ground.  The  multiple  frames  of  reference  for  the  participant  induced  spatial 
discordance  and  an  overall  decrease  in  tracking  performance  compared  to  trials  during  straight 
and  level  flight  and  ground  baseline  trials. 


Spatial  Discordance  3 


The  impact  of  conflicting  spatial  representations  in  airborne  unmanned  aerial  system  sensor 

control 

Emerging  concept  of  operations  (CONOPs)  for  Unmanned  Aerial  System  (UAS) 
employment  for  the  US  Navy  calls  for  the  potential  control  of  UAS  while  airborne,  such  as 
controlling  a  broad  area  maritime  surveillance  (BAMS)  UAS  from  an  airborne  P-8A  or  an 
unmanned  carrier-launched  surveillance  and  strike  (UCLASS)  UAS  from  a  fighter-class  aircraft. 
The  utility  of  pairing  these  two  types  of  aircraft  is  primarily  to  reduce  the  distance  of  the  UAS 
and  control  station,  thus  reducing  latency  of  control  input  and  increasing  bandwidth  for 
information  relay.  The  focus  of  this  project  is  to  assess  the  influence  of  disparate  spatial 
representations  on  the  human  operator  while  executing  sensor  (i.e.,  camera)  control. 

In  general,  performance  on  spatial  tasks  worsens  as  a  result  of  conflicting  and/or  inconsistent 
spatial  representations.  Shepard  &  Metzler  (1971)  used  3D  objects  to  test  peoples’  ability  to 
recognize  new  objects  across  different  viewpoints  and  noted  that  participants  performed  faster 
when  tested  with  the  original  learned  view  than  with  a  new  viewpoint.  Moreover,  they  found 
that  participants’  reaction  time  was  a  linear  function  of  the  angle  between  learned  and  presented 
view  -  the  greater  the  change  in  position,  the  slower  the  response.  This  oft-cited  “mental 
rotation”  task  implicates  some  cognitive  “cost”  to  mentally  rotate  an  object.  Could  the  mental 
processes  underpinning  the  rotation  task  be  similar  to  those  utilized  when  a  UAV  sensor  operator 
tracks  a  target?  We  predict  that  the  mental  processes  are  indeed  similar  and,  thus,  target  tracking 
performance  will  decline  while  in  an  airborne  environment. 

The  general  finding  of  performance  decrements  due  to  mental  manipulation  of  the  spatial 
orientation  of  objects  has  been  well-established  over  the  past  four  decades.  More  recently, 
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however,  it  has  been  determined  to  be  subject  to  further  distortions.  For  example,  Simons  & 
Wang  (1998)  discovered  that  the  ability  to  assess  the  spatial  configuration  of  objects  is  not 
merely  dependent  on  the  visual  stimuli.  They  found  that  a  viewpoint  change  does  not  impair  the 
understanding  of  where  objects  are  in  relation  to  each  other  unless  the  viewpoint  change  is 
decoupled  from  a  participant’s  control.  In  other  words,  when  a  participant  changed  their 
viewpoint  by  physically  changing  their  position,  their  ability  to  detect  subtle  changes  in  the 
spatial  configuration  of  objects  around  a  table  went  unchanged.  When  the  scene  was  moved 
independent  of  the  participant’s  control,  however,  the  ability  to  detect  changes  in  the  spatial 
layout  of  the  objects  was  impaired.  Similarly,  Wraga,  Creem-Regehr,  and  Proffitt  (2004)  found 
that  the  relative  position  of  objects  in  a  simulation  was  most  accurately  and  quickly  identified 
when  the  participant’s  motion  was  embedded  within  the  simulation.  (For  further  examples  of  the 
value  of  coupling  movement  and  viewpoint,  see  Jurgens  &  Becker,  2011;  Klatzky  et  al.,  1998; 
Rokos-Ewoldsen  et  al.,  1998;  Simons,  Wang,  &  Roddenbery,  2002;  Wraga,  Creem-Regehr,  & 
Proffitt,  2000;  and/or  see  Previc  &  Ercoline,  2004,  for  an  overview  of  spatial  disorientation  in 
aviation.) 

These  effects  have  been  shown  to  have  real-world  applicability  as  well.  Muth,  Walker,  and 
Fiorello  (2006)  demonstrated  that  controlling  one  vehicle  while  riding  in  another  reduced 
performance  in  their  measure  of  accuracy;  increased  latency  to  complete  their  task;  and  resulted 
in  a  four-fold  increase  in  motion  sickness.  This  study  used  ground  vehicles  rather  than  aircraft, 
but  clearly  indicates  that  discordant  spatial  cues  are  a  cause  for  concern.  Similar  findings  have 
been  discovered  for  tethered  Remotely  Piloted  Vehicles  (RPVs;  see  Hollands  &  Lamb,  201 1). 

Performance  decline  has  also  been  noted  for  aviation  tasks  in  spatially  discordant 
environments.  Reed  (1977)  demonstrated  that  simulated  turbulence  impaired  control  of  a  RPV. 
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Likewise,  Olson  and  colleagues  (2007)  demonstrated  that  discordance  between  the  platform 
motion  and  a  controlled  UAS  significantly  impairs  performance.  They  conducted  these 
experiments  controlling  a  UAS  simulator  on  a  motion-platform  (2006)  and  in  an  aircraft  (2007), 
with  vertical  axis  control  and  runway  alignment  impairments  evident  in  both  experiments. 
Additionally,  their  2007  results  suggest  that  the  presence  of  horizon  information  creates  greater 
impairment.  Their  methodology,  however,  was  limited  -  participants  were  only  seated  in  a 
forward  configured  seat  in  a  civilian  aircraft  and  only  rudimentary  maneuvers  were  used  (e.g., 
turn,  climb,  and  landing). 

Due  to  these  limitations,  the  current  experiment  included  multiple  seat  configurations  and  a 
highly  dynamic  flight  profile.  In  this  experiment,  participants  used  a  joystick  to  track  a  vehicle 
traversing  a  specified  path  around  the  airfield  at  NAS  Patuxent  River,  MD.  The  tracking 
crosshairs  were  depicted  via  a  “camera”  on  a  simulated  UAV  flying  an  elliptical  flight  pattern 
around  the  base  (see  Figure  1).  The  system  recorded  target  tracking  error  while  the  participants 
attempted  to  maintain  the  cursor  on  the  center  of  the  simulated  vehicle  as  it  “drove”  around  the 
base  (see  Figure  2).  In  a  counterbalanced  design,  trials  took  place  at  a  desk  (i.e.,  ground  trials) 
and  in  the  air  flying  a  figure-eight  flight  profile  while  quickly  ascending  and  descending  in  a  P-3 
(i.e.,  airborne  trials). 

The  current  experimental  environment  posed  a  unique  problem  to  the  visual  and  vestibular 
systems  of  the  participants.  Participants  operated  an  unmanned  aerial  system  sensor  simulator 
while  riding  in  the  aft  fuselage  of  a  large  military  aircraft  (i.e.,  P-3  Orion).  During  airborne 
trials,  the  aircraft  flew  alternating  straight-and-level  and  highly  dynamic  flight  profiles  to  induce 
variable  magnitudes  of  spatial  disparity  between  observed  visual  stimuli  and  perceived  vestibular 
information.  Consider  the  conflicting  vestibular  and  visuospatial  representations  among  the 


Spatial  Discordance  6 


movement  of  the  aircraft  and  ones’  own  position  within  the  aircraft;  the  simulated  UAS  position 
and  movement;  and  the  ventral  sensor  (i.e.,  camera)  position  and  orientation  on  the  UAS.  The 
translation  and  rotation  difficulties  within  the  aircraft  (e.g.,  heave,  sway,  surge,  pitch,  yaw,  roll) 
are  easily  and  quickly  resolved  by  most  people,  but  an  interesting  problem  arises  when  the 
participant  is  tasked  to  control  a  simulated  camera  on  a  simulated  UAS  that  is  in  a  different  flight 
profile  than  they  are  experiencing.  This  task  triples  the  degrees  of  freedom  that  must  be  resolved 
by  the  participant  -  the  aircraft,  UAS,  and  sensor  all  have  their  own  six  degrees  of  freedom. 

To  provide  an  example  of  multiple  degrees  of  freedom  for  this  experiment,  envision  the 
following:  An  aircraft  is  banking  left  and  descending;  the  simulated  UAS  is  banking  right  and 
ascending;  and  the  camera  is  slewing  aft  -  for  a  person  in  a  starboard  seat,  facing  towards  the 
center  of  the  aircraft,  great  discord  between  these  spatial  representations  and  their  relevant 
sensory  inputs  would  result.  This  creates  the  likelihood  not  only  of  noteworthy  momentary 
performance  lapses  until  these  spatial  discrepancies  are  reconciled,  but  also  pervasive  and 
sustained  fatigue  and  motion  sickness  -  not  to  mention  the  consequent  creation  of  significant 
safety  risks.  In  addition  to  the  behavioral  measures  previously  discussed,  we  administered  the 
motion  sickness  assessment  questionnaire  (MSAQ)  after  ground  trials  and  airborne  trials  to 
evaluate  the  effect  of  the  multiple  degrees  of  freedom  on  the  well-being  of  the  participants  (see 
Appendix  A). 

Although  this  operational  environment  is  quite  unique,  especially  for  experimentation,  it 
should  not  be  treated  any  different  from  other  experimental  environments  (Gibson,  1966; 
Stroffregen  &  Riccio,  1988).  In  other  words,  the  exceptionality  of  this  operational  environment 
elicits  behavior  specific  to  this  environment  as  a  result  of  the  perceptual  experience  and, 
although  unique,  still  provides  meaningful  information  about  conflicting  spatial  information 
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resolution  and  motor  behavior  in  a  tracking  task.  This  could  be  considered  a  limitation  of  the 
generalizability  of  the  current  study,  but  given  the  recent  rapid  advancement  of  technology  in 
aviation,  space,  and  underwater  environments,  value  can  be  found  through  this  examination  of  an 
infrequent  environmental  condition. 

Through  the  use  of  this  distinct  experimental  environment,  we  predict  that  when  participants 
are  airborne,  their  ability  to  track  the  target  with  the  simulated  UAV  camera  will  not  be  as 
accurate  or  consistent  via  several  measures.  First,  the  error  distance  between  the  cursor  and  the 
target  is  predicted  to  be  larger  when  participants  are  airborne  than  during  ground  trials.  Second, 
the  variability  of  the  cursor  movement  about  the  center  of  the  target  is  predicted  to  be  higher 
during  trials  in  the  plane.  Third,  the  amount  of  time  on  target  (TOT),  or  when  the  cursor  is 
within  five  meters  of  the  center  of  the  target,  is  predicted  to  be  longer  during  ground  trials  than 
during  airborne  trials.  Finally,  we  predict  that  self-reports  of  motion  sickness  will  indicate 
higher  magnitude  of  motion  sickness  during  air  trials  than  ground  trials. 

Method 

Participants 

Eight  Naval  Officers  participated  in  this  experiment.  Participants  had  current  flight 
physicals  and  were  qualified  to  ride  in  military  aircraft.  These  requirements  were  necessary 
given  that  data  collection  occurred  on  an  aircraft  during  flight.  All  efforts  to  recruit  an  equal 
number  of  male  and  female  participants  were  executed,  but  only  males  participated  in  this 
experiment  -  ages  ranged  from  33-49  (M  =  40.75,  SD  =  6.88).  Participants  read  and  signed 
consent  forms  approved  by  the  Naval  Air  Warfare  Center  -  Aircraft  Division  Institutional 
Review  Board  (NAWC-AD  IRB).  Potential  risks  and  their  mitigations  can  be  found  in 


Appendix  B. 
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Fortunately,  none  of  these  risks  or  discomforts  occurred  to  the  point  that  experimentation 
had  to  be  paused  or  cancelled.  Headaches  and  nausea  were  reported,  but  will  be  discussed  later 
in  the  results  section. 

Materials 

Four  networked  Dell  Precision  M6800  laptop  computers  (17.3”  screen)  presented  the 
UAS  sensor  simulators  to  the  participants.  Three  of  the  computers  ran  a  modified  version  of 
MetaVR™  v5.10  Scenario  Creation  Tool  (SCT),  which  is  a  3D  real-time  PC-based  virtual 
environment  generator.  This  software  was  modified  to  generate  the  sensor  information  (i.e., 
camera)  from  the  ventral  side  of  a  GlobalHawk  UAS.  The  flight  profile  approximated  that  of  the 
GlobalHawk,  and  maintained  an  unclassified  level  of  fidelity. 

The  fourth  computer  ran  the  MetaVR™  v5.10  Virtual  Reality  Scene  Generator  (VRSG). 
This  software  generates  surface  information  based  on  satellite  information.  Surface  information 
of  NAS  Patuxent  River,  MD,  was  chosen  because  all  participants  were  familiar  with  the 
geography  and  surface  structures  in  and  around  this  location.  The  fidelity  of  all  visual  stimuli 
approximated  that  of  current  (2014)  video  game  technology. 

Thrustmaster™  T-Flight  Hotas  X  Flight  Stick  joysticks  connected  to  the  three  participant 
computers.  The  default  transfer  function  of  the  joystick  to  the  cursor  on  the  screen  was  non¬ 
linear,  in  that,  small  movements  in  the  joystick  moved  the  cursor  to  a  lesser  extent  than  larger 
joystick  movements  in  a  non-linear  manner  (i.e.,  sigmoidal  transfer  function).  In  other  words, 
small  joystick  movements  would  not  noticeably  move  the  cursor,  whereas  long-distance  cursor 
movement  could  be  achieved  easily  without  moving  the  joystick  to  its  maximal  range. 

For  this  experiment,  a  Lockheed  P-3  Orion  provided  the  flight  environment.  The  P-3 
Orion  is  a  four-engine  turboprop  aircraft  developed  for  the  United  States  Navy.  Although 
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normally  used  for  anti-submarine  or  maritime  surveillance  missions,  the  aircraft  used  for  this 
experiment  had  a  nearly  empty  fuselage  rather  than  the  usual  configuration  for  military 
operations.  The  empty  fuselage  allowed  participants  to  walk  freely  in  between  trial  runs  and 
socialize  away  from  the  experimental  set-up  in  the  aft  area  (i.e.,  galley)  of  the  aircraft.  For 
inexperienced  readers,  the  length  of  the  aircraft  is  116’  10”,  leaving  ample  room  for  movement 
of  the  participants. 

Procedure 

In  a  counterbalanced  design,  participants  completed  ground  trials  either  before  or  after 
the  airborne  trials.  Due  to  the  small  sample  size,  initial  conditions  were  pseudo-randomly 
assigned  between  one  condition  and  the  other  in  the  order  that  participants  signed  up  for  the 
study  rather  than  through  truly  random  assignment. 

Regardless  of  initial  condition  assignment,  each  participant  was  trained  to  criterion  on 
one  of  the  laptop-based  UAS  sensor  operator  simulators.  Criterion  is  a  performance  measure  that 
is  based  on  an  individual’s  performance,  not  a  predefined  level  of  proficiency.  This  simulation 
training  occurred  on  the  ground  and  included  a  basic  introduction  to  the  system  displays  and 
controls  and  an  explanation  of  the  mission.  As  previously  mentioned,  the  mission  was  to  keep 
crosshairs  of  a  UAS  sensor  on  a  simulated  ground  vehicle  driving  a  pre-programmed  path  around 
a  local  military  base. 

For  ground  trials,  participants  chose  a  one-hour  time  period  that  suited  their  schedule 
within  one  week  prior  or  after  airborne  testing  depending  on  their  initial  condition  assignment. 
Participants  were  seated  in  front  of  one  of  the  UAS  sensor  operator  laptops  and  they  completed 
approximately  30-45  minutes  of  trials  that  lasted  approximately  1-2  minutes  each. 


Spatial  Discordance  10 


At  the  beginning  of  each  trial,  the  participant  began  with  their  “camera”  fully  zoomed-out 
in  an  attempt  to  prevent  participants  from  easily  replicating  motor  behavior  to  move  the  cursor 
over  the  target  vehicle.  Once  the  participant  indicated  that  they  were  ready  for  the  trial  to  begin, 
they  were  instructed  to  “zoom  in”  their  camera  to  find  the  target  vehicle  and  to  keep  the  cursor 
on  the  center  of  the  target  as  best  they  could.  When  the  participant  moved  the  cursor  over  the 
target  vehicle,  the  researcher  began  a  one-minute  timer  to  designate  the  duration  of  a  trial.  After 
a  minute  of  target  tracking  elapsed,  the  trial  was  over  and  the  participant  was  instructed  to  zoom 
out  their  camera  and  wait  for  the  next  trial  to  begin.  Upon  completion  of  the  ground  trials,  the 
participants  were  administered  the  MSAQ  for  comparison  with  the  airborne  trials.  We 
implemented  the  same  procedure  for  airborne  trials,  but  with  a  few  key  differences. 

After  take-off,  the  Principle  Investigator  set-up  three  test  stations  in  the  aft  area  of  the  P-3 
aircraft.  This  area  is  where  the  galley  (i.e.,  kitchen)  is  located  and  contains  bench  seating  at  a 
large  table  and  mounted  stool  seating  at  a  small  table.  This  seating  configuration  provided  space 
for  three  participants  to  be  run  at  a  time  in  three  different  seating  orientations:  forward, 
backward,  and  center  of  the  aircraft  facing  outboard.  In  addition  to  seating  configuration,  pilots 
flew  two  different  flight  profiles. 

The  first  flight  profile,  Profile  Alpha,  was  a  standard  racetrack  (i.e.,  oval)  profile  flown  at 
one  altitude.  The  second  flight  profile,  Profile  Beta,  was  a  figure  eight  path  flown  in  a  2000  feet 
per  minute  ascending  and  descending  pattern.  Two  groups  of  three  and  one  group  of  two 
participants  completed  30  trials  each.  The  Principal  Investigator  communicated  with  the  pilots 
via  electronic  means  to  indicate  when  to  change  from  Profile  Alpha  to  Profile  Beta  -  this 
manipulation  was  counterbalanced  among  the  groups  of  participants  as  well. 
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Data  Analyses 

Analyses  included  data  from  all  eight  participants  -  no  adverse  events  or  instructions 
misunderstandings  required  any  data  exclusions. 

Residual  distribution  and  link-function  assignment  to  transform  the  data  to  a  normal 
distribution  were  identified  with  Box-Cox  analysis,  and  planned  analyses  were  conducted  with 
linear-mixed  effects  modeling  (LME)  in  R  (available  at  www.r-project.org)  using  the  lme4 
package  (Nelder  &  Wedderbum,  1972).  Graphical  representations  of  the  data  were  produced  in 
R,  Microsoft  Excel,  and  JMP  v9,  a  graphical  analysis  product  developed  by  SAS. 

Two  within-subject  variables  (e.g.,  baseline/airborne  trials,  elapsed  time)  and  one 
between-subject  variable  (e.g.,  seat  configuration)  were  used  as  model  predictors  for  three 
dependent  variables  (e.g.,  tracking  error,  tracking  variability,  proportion  of  time  on  target). 
Student’s  t-test  analyses  of  the  MSAQ  data  were  conducted  as  well. 

Residual  distributions  for  tracking  data  were  positively  skewed,  indicating  that 
participants  were  more  often  near  the  target  than  far  away  from  the  target  in  both  airborne  and 
ground  trials  (see  Figures  3  &  4).  A  Box-Cox  analysis  indicated  that  a  modified  log-transform 
normalized  the  distributions  (see  Figures  5  &  6).  After  data  normalization,  LME  analyses 
revealed  that  the  experimental  manipulations  influenced  the  participants’  ability  to  track  targets. 
Results  are  written  in  a  manner  analogous  to  simple-effects  effects  tests  to  ease  interpretation  for 
the  reader. 

Results 

Flight  profile  influenced  tracking  performance  [t=l 8.83,  pc.OOl].  Post-hoc  analyses 
revealed  that  all  three  trial  types  resulted  in  significantly  different  tracking  performance  of  the 
participants.  Ground  trials  resulted  in  the  lowest  tracking  error,  followed  by  trials  conducted 
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during  Profile  Alpha,  and  trials  conducted  during  Profile  Beta  resulted  in  the  worst  tracking 
performance  of  the  participants  (see  Figure  7). 

Further  analyses  of  tracking  performance  revealed  that  seat  position  also  influenced 
tracking  performance  [t=-6.48,  pc.Ol].  Post-hoc  analyses  indicate  that  the  best  tracking 
performance  was  found  during  ground  trials,  forward  and  aft  seating  positions  were  significantly 
worse,  and,  surprisingly,  an  outboard-facing  orientation  resulted  in  the  worst  tracking 
performance  (see  Figure  8).  Although  we  did  not  assign  specific  hypotheses  to  seat 
configuration  as  a  predictor  of  performance,  these  results  suggest  that  further  investigation  is 
needed  and  will  be  discussed  further  in  the  next  section. 

Our  second  hypothesis  predicted  that  the  variability  of  cursor  movement  about  the  center 
of  the  target  would  be  higher  during  airborne  trials  than  ground  trials.  This  relationship  was 
found  to  be  true,  but  there  was  a  significant  interaction  between  trial  type  and  trial  number  for 
the  variability  of  tracking  performance  [t=- 18.01,  pc.OOl].  This  interaction  reveals  that  although 
performance  during  airborne  trials  began  more  variable  at  the  beginning  of  a  block  of  data 
collection,  performance  became  more  stable  as  the  experiment  progressed. 

Next,  we  predicted  that  participants  would  spend  less  time  on  target  (TOT)  during 
airborne  trials  than  during  ground  trials.  An  interaction  of  trial  type  and  trial  number  predicted 
TOT  performance  [F(2,2)  =  47.15.  pc.OOl].  As  predicted,  TOT  was  worse  during  airborne  trials, 
but  only  initially.  Similar  to  tracking  variability,  as  trials  progressed  TOT  increased  for  airborne 
trials.  Notably,  however,  TOT  performance  during  ground  trials  decreased  as  trials  progressed 
(see  Figure  10). 

Finally,  participants  indicated  that  symptoms  of  motion  sickness  were  more  prevalent 
during  airborne  trials  than  during  ground  trials  [F(l,14)  =  4.70,  pc.05]  (see  Figure  11). 
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Headaches,  fatigue,  and  nausea  were  the  most  common  symptoms  to  become  more  pronounced 
during  airborne  trials. 

Discussion 

This  experiment  investigated  the  effects  of  multiple  disparate  sources  of  spatial 
information  on  motor  tracking  behavior.  The  environment  in  which  this  experiment  occurred 
was  novel  and  highly  dynamic  causing  mismatched  information  from  the  visual  and  vestibular 
systems  to  negatively  influence  tracking  performance  and  overall  comfort  of  the  participants. 

Due  to  the  uniqueness  of  this  testing  environment,  some  may  ask  about  the  utility  of  the  project  - 
how  are  these  findings  extensible  or  applicable  in  other,  not  so  novel,  environments? 

At  first  glance,  one  may  be  inclined  to  side  with  the  previously  mentioned  skeptical 
reader,  but  after  reviewing  the  results  of  the  current  study,  this  section  will  outline  other  military 
applications  and  provide  further  questions  that  are  better  suited  for  the  laboratory.  The  results 
generally  supported  the  hypotheses  posited  for  this  study  and  some  unexpected  findings  came  to 
light  as  well. 

We  asserted  that  flight  condition  (i.e.,  ground/airborne)  would  influence  tracking 
performance  and  found  that  not  only  were  differences  in  performance  found  between  the 
baseline  (i.e.,  ground)  and  testing  (i.e.,  airborne)  trials,  but  the  phase  of  flight  (i.e.,  Profile  Alpha, 
Profile  Beta)  also  resulted  in  performance  differences.  These  results  suggest  that  not  only  does 
highly  dynamic  motion  negatively  influence  tracking  performance,  but  fairly  inert,  stable  motion 
can  similarly  lead  to  poorer  tracking  performance. 

During  analyses  of  tracking  performance,  we  noticed  fairly  obvious  differences  based  on 
seat  orientation.  Although  we  did  not  address  this  topic  as  an  original  hypothesis  due  to  a  lack  of 
existing  evidence  in  distinguishing  between  forward,  backward,  or  side-facing  orientation  in 
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performance  metrics,  we  decided  to  further  scrutinize  these  differences.  Similar  to  the  first 
hypothesis,  performance  during  ground  trials  was  better  than  during  airborne  trials,  but  facing 
outboard  during  testing  resulted  in  poorer  tracking  performance  than  facing  either  forward  or  aft. 
No  interactions  with  seat  orientation  were  found,  but  due  to  the  small  number  of  participants,  it 
would  not  be  surprising  to  find  more  interesting  results  that  could  elucidate  the  relationship  of 
motion  and  spatial  orientation  with  a  larger  number  of  participants. 

While  not  a  significant  difference,  outboard-facing  participants  tended  to  report  higher 
levels  of  headaches  and  nausea  than  forward  or  aft  facing  participants.  Perhaps  this  unnatural 
traversing  of  space  induces  more  spatial  discordance  than  more  natural  forward  or  backward 
motion.  To  the  uninformed  reader,  the  solution  to  this  issue  seems  simple  -  don’t  orient  seats 
facing  inboard  or  outboard.  It  should  be  noted,  however,  that  most  military  aircraft  have  side- 
oriented  payload  operator  seats  to  save  space  within  the  aircraft.  Therefore,  it  is  cost-prohibitive 
to  suggest  reconfiguration  of  seating  across  hundreds  of  aircraft  as  a  solution.  In  the  next  few 
paragraphs,  we  will  discuss  results  that  suggest  perhaps  training  or  repeated  exposure  to 
conflicting  spatial  information  may  result  in  performance  analogous  to  baseline  conditions, 
which  is  a  much  less  costly  solution. 

We  predicted  that  tracking  performance  would  be  more  variable  while  airborne  than 
during  ground  trials.  For  the  beginning  of  each  data  collection  session  while  airborne,  we  found 
this  relationship  to  be  true  -  performance  was  more  variable  while  airborne  than  during  ground 
trials.  As  time  progressed,  however,  performance  variability  returned  to  a  profile  similar  to  that 
of  baseline.  Would  this  initial  increase  in  variability  persist  over  time,  or  would  repeated 
exposure  eliminate  variable  tracking  performance? 
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Another  interaction  resulted  between  time  on  target  (TOT)  and  trial  number.  As 
previously  mentioned,  TOT  is  when  the  crosshair  was  within  5  meters  of  the  center  of  the 
tracking  target.  This  interaction  revealed  that  as  time  passed,  TOT  was  initially  highest  during 
ground  trials;  initially  lowest  during  Profile  Beta  (i.e.,  highly  dynamic);  and  TOT  was  stable 
during  Profile  Alpha  (i.e.,  not  dynamic).  Interestingly,  during  Profile  Beta,  performance  became 
less  variable  and  TOT  increased  as  the  experiment  progressed,  and  the  opposite  relationship  was 
found  to  be  true  for  ground  trials.  The  former  result  suggests  that  exposure  to  high  motion 
environments  while  performing  motor  skill  tasks  warrants  further  investigation.  Whereas  the 
latter  is  indicative  of  task  boredom;  a  more  thorough  investigation  of  this  performance  decrement 
should  be  considered  as  well. 

Finally,  as  expected,  participants  reported  higher  levels  of  motion  sickness  during 
airborne  flights  than  during  ground  trials.  Although  these  results  could  be  confounded  by 
demand  characteristics  of  the  experiment,  the  increased  trend  was  only  found  in  a  subset  of 
questions.  In  particular,  questions  involving  headache,  nausea,  and  fatigue  indicated  an  increase 
in  magnitude  while  other  questions  of  perspiration,  anxiety,  and  mood  remained  stable.  Since  an 
increase  across  all  measures  was  not  found,  we  can  conclude  that  participants  answered  the 
MSAQ  in  a  manner  consistent  with  their  self-assessment  and  not  due  to  demand  characteristics. 

Although  this  project  took  place  in  a  difficult  to  access  and  highly  specific  environment 
outside  of  the  laboratory,  the  lessons  learned  are  extensible  to  other  more  highly  controlled 
environments.  For  example,  pilot  studies  for  this  project  took  place  in  a  flight  simulator  with  a 
6-DOF  (i.e.,  degrees  of  freedom)  motion  base.  This  environment  provided  discrepant  vestibular 
and  visual  information  to  the  participants  without  requiring  access  to  a  large  experimental 
aircraft  -  the  results  of  the  pilot  studies  were  similar,  but  performance  degraded  to  a  smaller 
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magnitude.  We  expect  to  see  further  research  in  similar  environments  to  investigate  the  effect  of 
repeated  exposure  in  dynamic  motion  environments  to  evaluate  if  training  is  sufficient  to 
overcome  performance  changes  and  motion  sickness. 
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Appendix  A 

MOTION  SICKNESS  ASSESSMENT  QUESTIONNAIRE  (MSAQ) 

Instructions.  Using  the  scale  provided  above  each  statement,  please  rate  how  accurately  the 
following  statements  describe  your  experience. 

Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  sick  to  my  stomach. 

Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  faint-like. 


Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  annoyed/irritated. 

Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  sweaty. 


Not  at  all 

1 - 2 - 3 - 4 - 5 - 6 - 7- 

I  felt  queasy. 


Severely 
8 - 9 


Not  at  all 

1 - 2 - 3 - 4 - 5 - 6 - 7- 

I  felt  lightheaded. 


Severely 
8 - 9 


Not  at  all 

1 - 2 - 3 - 4 - 5 - 6 - 7- 

I  felt  drowsy. 


Severely 
8 - 9 


Not  at  all 

1 - 2 - 3 - 4 - 5 - 6 - 7- 

I  felt  clammy/cold  sweat. 


Severely 
8 - 9 


Not  at  all 
1 - 2- 


■3 - 4 - 5 - 6 - 7- 

I  felt  disoriented. 


Severely 
8 - 9 
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Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  tired/fatigued. 

Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  nauseated. 


Not  at  all 

1 - 2 - 3 - 4 - 5 - 6 - 7- 

I  felt  hot/warm. 


Severely 
8 - 9 


Not  at  all 

1 - 2 - 3 - 4 - 5 - 6 - 7- 

I  felt  dizzy. 


Severely 
8 - 9 


Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  like  I  was  spinning. 

Not  at  all  Severely 

1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 

I  felt  as  if  I  may  vomit. 


Not  at  all 
1 - 2- 


-3 - 4 - 5 - 6 - 7- 

I  felt  uneasy. 


Severely 
8 - 9 
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Appendix  B 

Several  potential  risks  and  discomforts  existed  for  participants.  These  risks  and 

discomforts  and  their  potential  mitigations  are  listed  below: 

Spatial  Disorientation  -  the  inability  to  correctly  interpret  aircraft  attitude,  altitude  or 
airspeed,  in  relation  to  the  Earth  or  point  of  reference,  especially  after  a  reference  point 
(e.g.,  the  horizon)  has  been  lost.  Spatial  disorientation  can  escalate  from  a  novel  and 
interesting  experience  (e.g.,  amusement  park  rides)  to  a  quite  uncomfortable  experience. 

If  a  participant  experienced  extreme  disorientation  to  the  point  of  discomfort,  the 
experiment  would  have  been  temporarily  stopped  while  the  participant  regained  a  point 
of  reference  to  regain  spatial  orientation.  If  they  chose  to  end  their  participation  in  the 
experiment,  the  aircraft  would  have  returned  to  base  and  testing  for  that  participant  would 
have  finished. 

Nausea  -  a  very  common  symptom  that  is  often  described  as  a  feeling  of  queasiness  or 
wooziness,  or  a  need  to  vomit.  If  a  participant  needed  to  vomit,  a  proper  receptacle 
would  have  been  provided.  The  experiment  would  have  been  temporarily  stopped  while 
the  participant  took  measures  to  relieve  their  nausea.  If  they  chose  to  end  their 
participation  in  the  experiment,  the  aircraft  would  have  returned  to  base  and  testing  for 
that  participant  would  have  finished. 

Hyperhidrosis  -  condition  characterized  by  abnormally  increased  sweating/  perspiration 
in  excess  of  that  required  for  regulation  of  body  temperature.  This  secondary  effect  of 
spatial  disorientation  and/or  nausea  is  benign  and  no  extraneous  prevention  or  treatment 
was  necessary. 

Headache  -  pain  anywhere  in  the  region  of  the  head  or  neck.  If  a  participant  experienced 
headache,  the  experiment  would  have  been  temporarily  stopped  while  the  participant  took 
measures  to  relieve  their  headache.  If  they  chose  to  end  their  participation  in  the 
experiment,  the  aircraft  would  have  returned  to  base  and  testing  for  that  participant  would 
have  finished. 

Flight  Mishap  (including  injury  or  death)  -  an  occurrence  associated  with  the  operation 
of  an  aircraft,  which  takes  place  between  the  time  any  person  boards  the  aircraft  with  the 
intention  of  flight  until  such  time  as  all  such  persons  have  disembarked,  where  a  person  is 
fatally  or  seriously  injured,  the  aircraft  sustains  damage  or  structural  failure  or  the  aircraft 
is  missing  or  is  completely  inaccessible.  All  safety  measures  and  training  were 
completed  by  all  aircrew  aboard  the  aircraft  prior  to  flight. 
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Figure  Captions 

Figure  1.  View  from  simulated  UAV  at  10,000’.  Trials  begin  in  “zoomed-out”  camera  position 
to  vary  cursor  starting  position  which  eliminates  rote  motor  movements. 

Figure  2.  “Zoomed- in”  view  of  trial  where  cursor  is  approximately  45°  off  target  in  the  center  of 
the  screen.  Note  that  Figure  2  is  a  zoomed-in  image  of  Figure  1. 

Figure  3.  Distribution  of  residuals  for  distance  from  target  for  airborne  trials  indicate  a  strong 
positive  skew  in  the  distribution.  This  distribution  suggests  participants  kept  the  cursor  closer  to 
the  target  for  more  time  than  farther  distances. 

Figure  4.  Log  transform  of  the  distribution  of  residuals  for  distance  from  target  for  airborne  trials 
normalizes  the  data  for  analyses. 

Figure  5.  Distribution  of  residuals  for  distance  from  target  for  ground  trials  indicate  a  strong 
positive  skew  in  the  distribution.  This  distribution  suggests  participants  kept  the  cursor  closer  to 
the  target  for  more  time  than  farther  distances. 

Figure  6.  Log  transform  of  the  distribution  of  residuals  for  distance  from  target  for  ground  trials 
normalizes  the  data  for  analyses. 

Figure  7.  Ground  trials  produced  the  least  error  distance  from  the  target  center.  Profile  Alpha 
(e.g.,  racetrack)  trials  led  to  significantly  worse  tracking  than  ground  trials,  and  Profile  Beta 
(e.g.,  dynamic)  trials  led  to  the  worst  average  tracking  performance  of  participants. 

Figure  8.  Inboard-facing  seat  orientation  resulted  in  the  worst  tracking  performance.  Forward 
and  aft  seating  resulted  in  significantly  better  performance,  and  ground  trials  resulted  in  the  best 
tracking  performance. 

Figure  9.  An  interaction  of  trial  type  and  trial  number  for  tracking  variability  indicates  that 
although  tracking  performance  is  initially  more  variable  during  airborne  trials,  it  stabilizes  and 
returns  to  baseline  variability  levels  as  trials  progress. 

Figure  10.  The  proportion  of  time  on  target  (i.e.,  within  5  meters)  was  determined  by  trial  type 
and  trial  number.  Performance  improved  for  participants  while  airborne,  but  degraded  during 
ground  trials  as  trials  progressed. 

Figure  11.  Participants  indicated  a  higher  magnitude  of  motion  sickness  symptoms  during 
airborne  trials  than  during  ground  trials. 
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